<!DOCTYPE html>
<html lang="zh-cn">
<head>
	
	<meta http-equiv="Content-Type" content="text/html;charset=utf-8">
	<title>
Tornado源码之Template | 
穷折腾</title>
	<link rel="stylesheet" href="/static/css/style.css" />
	<link rel="stylesheet" href="/static/css/pygments.css" />
	<link rel="alternate" type="application/rss+xml" title="RSS" href="http://blog.zorro.im/rss.xml" />
	<link rel="icon" type="image/png" href="/static/img/favicon.png">
</head>
<body>
    <div id="container">
      <div id="main" role="main">
        <header>
		<h1>
Tornado源码之Template
</h1>
		</header>

     <nav>
		<span><a title="home page" class="" href="/">home</a></span>
		<span><a title="标签" class="" href="/tags/index.html">标签</a></span>
		<span><a title="订阅" class="" href="/rss.xml">订阅</a></span>
        <span><a href="/posts/about.html" title="关于">关于</a></span>
     </nav>

     <article class="content">
        <section class="post">
            
		<p>Tornado底层的源码分析已经有很多人写过了，他们的水平都在我之上，写的也比我好，所以我就不再重复了。打算剑走偏锋，研究一下tornado周边的东西。这篇就研究一下之前一直很感兴趣的Template。</p>
<p>Template的工作流程如下：</p>
<ol>
<li>读取模板文件，解析成相应的数据结构</li>
<li>把解析到的结构拼接成Python代码</li>
<li>将生成的代码编译成字节码</li>
<li>执行字节码，返回结果</li>
</ol>
<p>（这其中还会有一些编码转换、特殊字符转义等工作，本文中不做研究。）</p>
<h2>模板解析与代码生成</h2>
<p>步骤1-3主要由Template类的<strong>init</strong>方法完成，关键代码如下：</p>
<div class="codehilite"><pre><span class="n">reader</span> <span class="o">=</span> <span class="n">_TemplateReader</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">escape</span><span class="o">.</span><span class="n">native_str</span><span class="p">(</span><span class="n">template_string</span><span class="p">))</span>
<span class="bp">self</span><span class="o">.</span><span class="n">file</span> <span class="o">=</span> <span class="n">_File</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">_parse</span><span class="p">(</span><span class="n">reader</span><span class="p">,</span> <span class="bp">self</span><span class="p">))</span>
<span class="bp">self</span><span class="o">.</span><span class="n">code</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_generate_python</span><span class="p">(</span><span class="n">loader</span><span class="p">,</span> <span class="n">compress_whitespace</span><span class="p">)</span>
</pre></div>


<p>首先创建了一个reader，用来读取模板字符串。<code>_TemplateReader</code>这个类还重载了一些诸如<code>__getitem__</code>这样的方法，可以很方便的来操作字符串。</p>
<p><code>_parse</code>这个函数负责了模板的语法的解析和数据结构的生成。在这个过程中，会将<code>{{title}}</code>这样的字符串解析成<code>_Expression</code>，<code>{% for line in lines %}</code>解析成<code>_IntermediateControlBlock</code>等。这些类都继承自<code>_Node</code>。解析的最后结果是返回一个由这些类为节点组成的树状结构，树的根节点是<code>_File</code>。</p>
<p>当字符串解析完成之后，调用Template的<code>_generate_python</code>方法，来生成Python代码。代码生成时会调用每个树节点的<code>generate</code>方法。</p>
<p><code>_File</code>的<code>generate</code>代码如下：</p>
<div class="codehilite"><pre><span class="k">def</span> <span class="nf">generate</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">writer</span><span class="p">):</span>
    <span class="n">writer</span><span class="o">.</span><span class="n">write_line</span><span class="p">(</span><span class="s">&quot;def _tt_execute():&quot;</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">line</span><span class="p">)</span>
    <span class="k">with</span> <span class="n">writer</span><span class="o">.</span><span class="n">indent</span><span class="p">():</span>
        <span class="n">writer</span><span class="o">.</span><span class="n">write_line</span><span class="p">(</span><span class="s">&quot;_tt_buffer = []&quot;</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">line</span><span class="p">)</span>
        <span class="n">writer</span><span class="o">.</span><span class="n">write_line</span><span class="p">(</span><span class="s">&quot;_tt_append = _tt_buffer.append&quot;</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">line</span><span class="p">)</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">body</span><span class="o">.</span><span class="n">generate</span><span class="p">(</span><span class="n">writer</span><span class="p">)</span>
        <span class="n">writer</span><span class="o">.</span><span class="n">write_line</span><span class="p">(</span><span class="s">&quot;return _tt_utf8(&#39;&#39;).join(_tt_buffer)&quot;</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">line</span><span class="p">)</span>
</pre></div>


<p>这个方法会会生成类似这样的语句：</p>
<div class="codehilite"><pre><span class="k">def</span> <span class="nf">_tt_execute</span><span class="p">():</span>
    <span class="n">_tt_buffer</span> <span class="o">=</span> <span class="p">[]</span>
    <span class="n">_tt_append</span> <span class="o">=</span> <span class="n">_tt_buffer</span><span class="o">.</span><span class="n">append</span>

    <span class="c">#body</span>

    <span class="k">return</span> <span class="n">_tt_utf8</span><span class="p">(</span><span class="s">&#39;&#39;</span><span class="p">)</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">_tt_buffer</span><span class="p">)</span> 
</pre></div>


<p>由于<code>_File</code>是这棵树的根，所以最终生成的代码都会被嵌套在一个叫<code>_tt_execute</code>的函数内，而这个函数内所有语句生成的字符串会被塞到_tt_buffer中被返回。</p>
<p>注意一下这行代码<code>with writer.indent():</code>，先看一下源码：</p>
<div class="codehilite"><pre><span class="k">def</span> <span class="nf">indent</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
    <span class="k">class</span> <span class="nc">Indenter</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
        <span class="k">def</span> <span class="nf">__enter__</span><span class="p">(</span><span class="n">_</span><span class="p">):</span>
            <span class="bp">self</span><span class="o">.</span><span class="n">_indent</span> <span class="o">+=</span> <span class="mi">1</span>
            <span class="k">return</span> <span class="bp">self</span>

        <span class="k">def</span> <span class="nf">__exit__</span><span class="p">(</span><span class="n">_</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">):</span>
            <span class="k">assert</span> <span class="bp">self</span><span class="o">.</span><span class="n">_indent</span> <span class="o">&gt;</span> <span class="mi">0</span> 
            <span class="bp">self</span><span class="o">.</span><span class="n">_indent</span> <span class="o">-=</span> <span class="mi">1</span>

    <span class="k">return</span> <span class="n">Indenter</span><span class="p">()</span>
</pre></div>


<p>重载了<code>__enter__</code>和<code>__exit__</code>方法，这样每次在<code>with writer.indent()</code>代码段里面，缩进就自动加一，退出来之后缩进就自动减一。</p>
<p>在这里举个简单的例子来看一下生成的代码：</p>
<div class="codehilite"><pre><span class="kn">from</span> <span class="nn">tornado.template</span> <span class="kn">import</span> <span class="n">Template</span>

<span class="n">html</span> <span class="o">=</span> <span class="s">&#39;&#39;&#39;</span><span class="se">\</span>
<span class="s">&lt;ul&gt;</span>
<span class="s">{</span><span class="si">% f</span><span class="s">or l in lines %}</span>
<span class="s">    &lt;li&gt;{{l}}&lt;/li&gt;</span>
<span class="s">{</span><span class="si">% e</span><span class="s">nd %}</span>
<span class="s">&lt;/ul&gt;</span>
<span class="s">&#39;&#39;&#39;</span>

<span class="n">t</span> <span class="o">=</span> <span class="n">Template</span><span class="p">(</span><span class="n">html</span><span class="p">)</span>
<span class="k">print</span><span class="p">(</span><span class="n">t</span><span class="o">.</span><span class="n">code</span><span class="p">)</span>
</pre></div>


<p>执行完这段代码打印结果如下：</p>
<div class="codehilite"><pre><span class="k">def</span> <span class="nf">_tt_execute</span><span class="p">():</span>  <span class="c"># &lt;string&gt;:0</span>
    <span class="n">_tt_buffer</span> <span class="o">=</span> <span class="p">[]</span>  <span class="c"># &lt;string&gt;:0</span>
    <span class="n">_tt_append</span> <span class="o">=</span> <span class="n">_tt_buffer</span><span class="o">.</span><span class="n">append</span>  <span class="c"># &lt;string&gt;:0</span>
    <span class="n">_tt_append</span><span class="p">(</span><span class="s">&#39;&lt;ul&gt;</span><span class="se">\n</span><span class="s">&#39;</span><span class="p">)</span>  <span class="c"># &lt;string&gt;:2</span>
    <span class="k">for</span> <span class="n">l</span> <span class="ow">in</span> <span class="n">lines</span><span class="p">:</span>  <span class="c"># &lt;string&gt;:2</span>
        <span class="n">_tt_append</span><span class="p">(</span><span class="s">&#39;</span><span class="se">\n</span><span class="s">    &lt;li&gt;&#39;</span><span class="p">)</span>  <span class="c"># &lt;string&gt;:3</span>
        <span class="n">_tt_tmp</span> <span class="o">=</span> <span class="n">l</span>  <span class="c"># &lt;string&gt;:3</span>
        <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">_tt_tmp</span><span class="p">,</span> <span class="n">_tt_string_types</span><span class="p">):</span> <span class="n">_tt_tmp</span> <span class="o">=</span> <span class="n">_tt_utf8</span><span class="p">(</span><span class="n">_tt_tmp</span><span class="p">)</span>  <span class="c"># &lt;string&gt;:3</span>
        <span class="k">else</span><span class="p">:</span> <span class="n">_tt_tmp</span> <span class="o">=</span> <span class="n">_tt_utf8</span><span class="p">(</span><span class="nb">str</span><span class="p">(</span><span class="n">_tt_tmp</span><span class="p">))</span>  <span class="c"># &lt;string&gt;:3</span>
        <span class="n">_tt_tmp</span> <span class="o">=</span> <span class="n">_tt_utf8</span><span class="p">(</span><span class="n">xhtml_escape</span><span class="p">(</span><span class="n">_tt_tmp</span><span class="p">))</span>  <span class="c"># &lt;string&gt;:3</span>
        <span class="n">_tt_append</span><span class="p">(</span><span class="n">_tt_tmp</span><span class="p">)</span>  <span class="c"># &lt;string&gt;:3</span>
        <span class="n">_tt_append</span><span class="p">(</span><span class="s">&#39;&lt;/li&gt;</span><span class="se">\n</span><span class="s">&#39;</span><span class="p">)</span>  <span class="c"># &lt;string&gt;:4</span>
        <span class="k">pass</span>  <span class="c"># &lt;string&gt;:2</span>
    <span class="n">_tt_append</span><span class="p">(</span><span class="s">&#39;</span><span class="se">\n</span><span class="s">&lt;/ul&gt;</span><span class="se">\n</span><span class="s">&#39;</span><span class="p">)</span>  <span class="c"># &lt;string&gt;:6</span>
    <span class="k">return</span> <span class="n">_tt_utf8</span><span class="p">(</span><span class="s">&#39;&#39;</span><span class="p">)</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">_tt_buffer</span><span class="p">)</span>  <span class="c"># &lt;string&gt;:0</span>
</pre></div>


<p>生成的每条语句后都会有该条语句在模板文件中的行号。</p>
<p>代码生成之后，就调用python的<code>compile</code>函数，将代码编译成了字节码。</p>
<h2>模板渲染</h2>
<p>当执行Template的<code>generate</code>方法时，首先会把一些常用函数比如<code>datetime</code>等以及用户输入的参数放到namespace中，作为执行字节码时的全局命名空间。</p>
<p>这是执行字节码的代码：</p>
<div class="codehilite"><pre><span class="n">exec_in</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">compiled</span><span class="p">,</span> <span class="n">namespace</span><span class="p">)</span>
<span class="n">execute</span> <span class="o">=</span> <span class="n">namespace</span><span class="p">[</span><span class="s">&quot;_tt_execute&quot;</span><span class="p">]</span>
<span class="n">linecache</span><span class="o">.</span><span class="n">clearcache</span><span class="p">()</span>
<span class="k">return</span> <span class="n">execute</span><span class="p">()</span>
</pre></div>


<p>其中，<code>exec_in</code>的定义如下：</p>
<div class="codehilite"><pre><span class="k">if</span> <span class="n">sys</span><span class="o">.</span><span class="n">version_info</span> <span class="o">&gt;</span> <span class="p">(</span><span class="mi">3</span><span class="p">,):</span>
    <span class="k">exec</span><span class="p">(</span><span class="s">&quot;&quot;&quot;</span>
<span class="s">def raise_exc_info(exc_info):</span>
<span class="s">    raise exc_info[1].with_traceback(exc_info[2])</span>

<span class="s">def exec_in(code, glob, loc=None):</span>
<span class="s">    if isinstance(code, str):</span>
<span class="s">        code = compile(code, &#39;&lt;string&gt;&#39;, &#39;exec&#39;, dont_inherit=True)</span>
<span class="s">    exec(code, glob, loc)</span>
<span class="s">&quot;&quot;&quot;</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
    <span class="k">exec</span><span class="p">(</span><span class="s">&quot;&quot;&quot;</span>
<span class="s">def raise_exc_info(exc_info):</span>
<span class="s">    raise exc_info[0], exc_info[1], exc_info[2]</span>

<span class="s">def exec_in(code, glob, loc=None):</span>
<span class="s">    if isinstance(code, basestring):</span>
<span class="s">        # exec(string) inherits the caller&#39;s future imports; compile</span>
<span class="s">        # the string first to prevent that.</span>
<span class="s">        code = compile(code, &#39;&lt;string&gt;&#39;, &#39;exec&#39;, dont_inherit=True)</span>
<span class="s">    exec code in glob, loc</span>
<span class="s">&quot;&quot;&quot;</span><span class="p">)</span>
</pre></div>


<p>这里用到了一个技巧，用<code>exec</code>来根据python版本动态的添加函数。</p>
<p>从上面的例子中可以看出，执行完这段字节码，namespace中会创建一个名为<code>_tt_execute</code>函数，然后再执行这个函数，得到的结果就是模板渲染完的结果。</p>
<p>至此，Template的工作流程就介绍完了~</p>
	</section>
	<section class="meta">
		<span class="tags">Tagged by 
			<a href="/tags/tornado.html">tornado</a>
			<a href="/tags/template.html">template</a>
		</span>

		<span class="time">&nbsp;<time datetime="2013-11-21">2013-11-21</time></span>
	</section>
	<!-- Duoshuo Comment BEGIN -->
<div class="ds-thread"></div>
<script type="text/javascript">
	var duoshuoQuery = {short_name:"zqqf16"};
	(function() {
		var ds = document.createElement('script');
		ds.type = 'text/javascript';ds.async = true;
		ds.src = 'http://static.duoshuo.com/embed.js';
		ds.charset = 'UTF-8';
		(document.getElementsByTagName('head')[0] 
		|| document.getElementsByTagName('body')[0]).appendChild(ds);
	})();
</script>
<!-- Duoshuo Comment END -->

<hr/>


        </section>
     </article>
	 <div id="copy">&copy; Powered by <a href="https://github.com/zqqf16/zqqf16.github.com" title="Peanut">Peanut</a> | Themed by <a href="http://lhzhang.com" title="sext ii">sext ii</a></div>
      </div>
    </div> <!--! end of #container -->
	<script>
  		(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
  			(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
  		 m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
  			})(window,document,'script','//www.google-analytics.com/analytics.js','ga');

  		ga('create', 'UA-41282906-2', 'zorro.im');
  		ga('send', 'pageview');
	</script>
</body>
</html>
